Traffic on broadband networks continues to grow, such as driven by video content. Two contributors to video traffic growth are the bit-rate growth of individual streams as their resolution increases and the growth in the number of unique streams carried by a network as broadcast content consumption is replaced by unicast content consumption. Specifically, as more users (customers, subscribers, etc.) “cut the cord,” they resort to streaming content from various Over-the-Top (OTT) content providers. Of course, there is other content which contributes to the growth. Often the content, including the video content, is sourced from the Internet, outside of service providers' network domains. Service providers have been unable to increase Internet access revenue as average bandwidth has grown, so to maintain profitability they need to reduce their network costs continually. Conventionally, service providers have a number of solutions available to increase access bandwidth and reduce the cost per bit of broadband service delivery. A number of these conventional solutions are based on fundamental improvements in the capacity and cost of transmission and switching technology, and have resulted in orders of magnitude improvement in the cost/bit for delivering broadband services in the past. However, exponentials have a way of catching up, and service providers are now experiencing more difficulty in delivering sufficient network cost savings relying strictly on equipment cost savings, and need to seek fundamental architectural change.
There are various conventional approaches to the scalability and cost problems service providers face.
First, transmission capacity has scaled through Dense Wave Division Multiplexing (DWDM) and spectral efficiency improvements which provide conventional systems on the order of 10 Tb/s capacity, with an additional factor of 2 to 3× for metro distances. However, conventional DWDM transmission technology, limited in the spectrum by the optical amplifier passband, is approaching Shannon's limit. Increasing transmission symbol constellation complexity increases spectral efficiency and therefore system capacity, but maximum transmission distance becomes severely limited, even for the relatively short distances in the metro.
Second, switching capacity has scaled with relative simple packet switching circuitry (e.g., Application-Specific Integrated Circuits (ASICs) approaching 3.2 Tb/s chip input/output (I/O). More complex and more scalable switching systems, such as Internet Protocol (IP) routers, with the ability to manage large numbers of flows, require more complex ASICs and buffer memory and come with significantly higher cost and power. For example, at the time of writing, simple packet switching ASICs targeted at data center top-of-rack (TOR) switching are approaching I/O bandwidth of 3.2 Tb/s and fit in a 1 Rack Unit (RU) switch form factor, primarily limited by the I/O faceplate density. The platforms service providers require, because of requirements on the granularity of flow management, modularity, scalability, and built-in redundancy, have much lower density, and, therefore, higher cost/bit. IP routers are even less dense, and, therefore, more costly than packet-optical solutions. As switching ASICs increase in scale their cost/bit improves, but their total power dissipation increases, which presents a density and, therefore, cost challenge.
Third, there is a trend towards network convergence with the functional integration of transmission and switching. This integration can include Packet-Optical Transport Systems (POTS), IP/DWDM integration, etc. The integration seeks to remove external interfaces thereby reducing network cost. However, the integration does compromise overall density, i.e., integration of coherent transmission in metro switching platforms, either POTS or IP/DWDM, limits the switching system density that can be achieved, and presents a further cost and scalability challenge.
Fourth, there is a trend towards higher-level functional integration such as where switches and routers include integrated devices supporting such functions as distributed transparent content caching, Broadband Network Gateway (BNG), and servers to host virtual network functions. The value here is the removal of separate pieces of networking, processing, and storage equipment by reassigning functions to integrated host devices. For example, BNG modules are added to router platforms for subscriber management applications in some service provider networks. Transparent caches for popular video content have been widely studied but have proven less cost-effective the closer to the edge of the network they are deployed. These solutions are costly to implement and are not able to scale efficiently with the growth in user-to-content traffic demand. Most caching continues to prove effective in centralized locations, such as major Internet peering centers or hub data centers in Tier 1, 2 or 3 markets.
Fifth, network elements forming service provider networks are trending towards commoditization. Service providers are picking up on a trend started by the operators of mega-scale data centers to procure “white-box” switches or “bare metal” switches and servers. This commoditization has led to an effort to develop designs for open-source switching hardware.
Sixth, Software Defined Networking (SDN) is evolving to separate control and data planes, and this has significant momentum among service providers. SDN has also been applied to the optical layer.
Finally, network functions are being virtualized through Network Function Virtualization (NFV). Since, server processing power and I/O bandwidth continue to increase, many network functions now residing in stand-alone network appliances that are hosted on virtual machines within processors in a data center. Besides hardware cost savings, this trend brings substantial operational costs savings.
The approaches above can be applied within the context of the existing network architecture, with service providers replacing network components, as improved versions are available. Broadband access networks are inherently distributed entities, distributing and collecting traffic from hundreds of thousands to millions of endpoints in a major market (i.e., residences, businesses and mobile towers). This is achieved by a number of tiers of aggregation/distribution switches between the user and a regional gateway to the Internet. The first six items in the list above can be applied to the switches and transmission links in this distributed network. The seventh item above, NFV, could be applied as a box-by-box replacement for some functions in the distributed access/aggregation network described above, but greater economies of scale and operational simplicity can be achieved by placing them in centralized data centers within a market area. Limiting the number of these data centers to a small fraction of the number of central offices in use today reduces the number of physical locations housing complex equipment to manage, greatly reducing operational costs.
Thus, generally speaking, service providers are expected to maintain the same transport and switching architecture in the Wide Area Network (WAN) network, but plan to replace a number of network functions with virtual network functions hosted in data centers. Conventionally, IP routers perform an integrated aggregation and IP routing function. These IP routers typically operate over separate optical transport equipment. Further, existing data centers are currently operated independently from the metro or wide area networks (MAN or WANs), resulting in inefficient handoff between the network and the data center, typically at the expensive IP network layer.
One innovation in data center architecture that has some relevance to the problem described here is the hybrid packet-optical architecture. There are conventional approaches to hybrid packet-optical data center networks that are not new. For example, the Helios architecture described a hybrid packet-optical solution to enable high-capacity optical links between top-of-rack switches (Farrington, Nathan, et al. “Helios: a hybrid electrical/optical switch architecture for modular data centers” ACM SIGCOMM Computer Communication Review 41.4 (2011): 339-350). More recently, this approach has been developed as a commercial offering called Optical Aid (by Calient and Packetcounter). Plexxi also promotes a commercial data center network fabric that combines packet and optical (electrical cross-point) switches to enable dynamically reprogrammable interconnection between changing communities of interest that Plexxi calls Affinities. Ericsson's SOLuTIoN (SDN-based OpticaL Traffic steering for NFV) architecture also promotes a hybrid packet-optical approach. In Ericsson's design, they separate servers into two groups that are accessed via either (i) an optical switch for high-bandwidth traffic or (ii) a packet switch for fine granularity traffic. In this approach, an aggregation router located outside the data center is used to pre-sort IP-routed user traffic into the two groups associated with packet switching and optical switching inside the data center.
Typically, when optical switches are considered for use in a data center, such as described in the Helios approach, the application is to support large flows (sometimes described as ‘elephant’ flows) that would disrupt the efficiency of the existing packet network. The research challenge, as yet unsolved, is the automatic detection and timely configuration of optical resources in a general-purpose data center. The Plexxi solution, which implements a Helios-type architecture, does not solve the problem of sensing flows, but depends on the modeling of data center workflows to determine customized interconnection architectures. Following this modeling, incremental links between packet switches are configured to allow improved performance and efficiency in an interconnect configuration that is otherwise blocking. It is costly to implement fully non-blocking interconnects within large data centers. Also, in all cases, the optical switches are located in the data center and are not implemented as a distributed switch across the WAN.